-- Vision-Language-Action (VLA) models have emerged as a prominent research area, showcasing significant potential across a variety of applications. However, their performance in endoscopy robotics, particularly endoscopy capsule robots that perform actions within the digestive system, remains unexplored. The integration of VLA models into endoscopy robots allows more intuitive and efficient interactions between human operators and medical devices, improving both diagnostic accuracy and treatment outcomes. By processing interleaved visual inputs, and textual instructions, CapsDT can infer corresponding robotic control signals to facilitate endoscopy tasks. In addition, we developed a capsule endoscopy robot system, a capsule robot controlled by a robotic arm-held magnet, addressing different levels of four endoscopy tasks and creating corresponding capsule robot datasets within the stomach simulator . Comprehensive evaluations on various robotic tasks indicate that CapsDT can serve as a robust vision-language generalist, achieving state-of-the-art performance in various levels of endoscopy tasks while achieving a 26.25% success rate in real-world simulation manipulation. I. INTRODUCTION Endoscopy, for both diagnostic and therapeutic interventions, provides direct visualization and treatment capabilities within the gastrointestinal (GI) tract [1], [2], [3].

Magnetic resonance imaging (MRI)-guided robotic systems offer great potential for new minimally invasive medical tools, including MRI-powered miniature robots. By re-purposing the imaging hardware of an MRI scanner, the magnetic miniature robot could be navigated into the remote part of the patient's body without needing tethered endoscopic tools. However, the state-of-art MRI-powered magnetic miniature robots have limited functionality besides navigation. Here, we propose an MRI-powered magnetic miniature capsule robot benefiting from acoustic streaming forces generated by MRI-guided high-intensity focus ultrasound (HIFU) for controlled drug release. Our design comprises a polymer capsule shell with a submillimeter-diameter drug-release hole that captures an air bubble functioning as a stopper. We use the HIFU pulse to initiate drug release by removing the air bubble once the capsule robot reaches the target location. By controlling acoustic pressure, we also regulate the drug release rate for multiple location targeting during navigation. We demonstrated that the proposed magnetic capsule robot could travel at high speed up to 1.13 cm/s in ex vivo porcine small intestine and release drug to multiple target sites in a single operation, using a combination of MRI-powered actuation and HIFU-controlled release. The proposed MRI-guided microrobotic drug release system will greatly impact minimally invasive medical procedures by allowing on-demand targeted drug delivery.

At the beginning of the new millennia, wireless capsule endoscopy was introduced as a minimally invasive method of inspecting the digestive tract. The possibility of collecting images deep inside the human body just by swallowing a "pill" revolutionized the field of gastrointestinal endoscopy and sparked a brand-new field of research in robotics: medical capsule robots. These are self-contained robots that leverage extreme miniaturization to access and operate in environments that are out of reach for larger devices. In medicine, capsule robots can enter the human body through natural orifices or small incisions, and detect and cure life-threatening diseases in a non-invasive manner. This talk provides a perspective on how this field has evolved in the last ten years.

A host of bizarre biomedical robots turned up at ICRA 2017, IEEE's flagship robotics conference, which took place earlier this month in Singapore. We saw swallowable robots that poke the stomach with needles and worm-like robots that explore the colon. Equal parts unnerving and fascinating, these bots aim to help people--perhaps in ways we hope we never need. This capsule robot innocuously tumbles around inside your stomach--until it reaches suspicious-looking tissue. Then, like an EpiPen on steroids, the soft-bodied bot whips out a needle and jabs that spot inside your stomach in ten fast pumping movements.

A host of bizarre biomedical robots turned up at ICRA 2017, IEEE's flagship robotics conference, which took place earlier this month in Singapore. We saw swallowable robots that poke the stomach with needles and worm-like robots that explore the colon. Equal parts unnerving and fascinating, these bots aim to help people--perhaps in ways we hope we never need. This capsule robot innocuously tumbles around inside your stomach--until it reaches suspicious-looking tissue. Then, like an EpiPen on steroids, the soft-bodied bot whips out a needle and jabs that spot inside your stomach in ten fast pumping movements.

Could tiny robots change the way doctors perform colonoscopies? Researchers are touting an autonomous robot that can explore someone's colon to help catch cancers and lesions in that body part and remove them. After successfully testing the robot numerous times in a pig's colon, the team presented its findings at the Digestive Disease Week conference, describing the capsule machine as able to do "intricate maneuvers," according to the American Gastroenterological Association. "Researchers believe this technology will reduce the potential discomfort of colonoscopies and lead to more people undergoing the life-saving screening test." The 18-millimeter capsule robot is still inserted through the rectum and still has a tether, but it is a soft one and smaller than the endoscopes doctors currently use to explore a patient's colon, which has the added benefit of allowing for other instruments like for irrigation and suction.

Despite its potential to save lives, many people fear the discomfort of having a colonoscopy. But in the future, tiny robots could be deployed to search for pre-cancerous lesions and tumours in the bowel, resulting in less discomfort for patients. While such robots sound like they belong in a sci-fi novel, researchers have shown an 18mm magnetised capsule colonoscope can perform intricate and sometimes autonomous movements inside the colon for the first time. Researchers have shown an 18mm magnetised capsule colonoscope (right) can perform intricate and sometimes autonomous movements inside the colon for the first time. The'capsule robot' was guided by an external magnet attached to a robotic arm (left) The'capsule robot' was paired with standard medical instruments and was guided by an external magnet attached to a robotic arm.